A Generic Nearest Point Algorithm for Solving Support Vector Machines

We propose a nearest point algorithm (NPA) for optimization of a family of Support Vector Machines (SVM) methods. Geometrically, optimizing SVM corresponds to finding the nearest points between two polyhedrons. In classification, the hard margin case corresponds to finding the closest points of the convex hull of each class. The soft margin case corresponds to finding the closest points in the reduced convex hulls. A prior nearest point algorithm (NPA) was very effective but limited to the convex hull case for classification. We propose a generic NPA applicable to both the regular and reduced convex hull cases. Our approach is applicable to both classification and regression SVM based on 1-norm and 2-norm error functions. The resulting algorithm is efficient, easy to implement and requires no optimization solvers. We review how SVM regression with 2-insensitive loss can be regarded as a nearest point problem for two reduced convex hulls. Experimental results for NPA for SVM classification problems indicate the method is extremely promising. We discuss extensions of the approach applicable to very large databases.